Eecient Algorithms for Online Decision Problems

In an online decision problem, one makes a sequence of decisions without knowledge of the future. Tools from learning such as Weighted Majority and its many variants 13, 18, 4] demonstrate that online algorithms can perform nearly as well as the best single decision chosen in hindsight, even when there are exponentially many possible decisions. However, the naive application of these algorithms is inee-cient for such large problems. For some problems with nice structure, specialized eecient solutions have been developed 10, 16, 17, 6, 3]. We show that a very simple idea, used in Hannan's seminal 1957 paper 9], gives eecient solutions to all of these problems. Essentially, in each period, one chooses the decision that worked best in the past. To guarantee low regret, it is necessary to add randomness. Surprisingly, this simple approach gives additive regret per period, eeciently. We present a simple general analysis and several extensions, including a (1+)-competitive algorithm as well as a lazy one that rarely switches between decisions.